Add metadata at item level for Disasters (event name and monty-stac schema)#415
Open
kyle-lesinger wants to merge 11 commits intodevfrom
Open
Add metadata at item level for Disasters (event name and monty-stac schema)#415kyle-lesinger wants to merge 11 commits intodevfrom
kyle-lesinger wants to merge 11 commits intodevfrom
Conversation
Member
Author
|
@smohiudd here is the first part of the new updates for Disasters. If we fully incorporate monty stac extension, I'll have to scrape more properties from the filenames. But that is TBD. |
|
Just wanted to jump in here - a few weeks ago I brought up the monty extension to see if there is anything we can learn on best practices for having to deal with event based and running catalogs of data. This is not a request from the disasters team (who probably have never heard of this extension) and I would want us to include it only if we have a good understanding of the added value (i.e. does this make the data more easily searchable in other disasters catalogs?) |
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
In order to effectively filter through data within veda-ui, additional property information is needed at the item level. This will assist with restricting data within the E&A to only data that is relevant for a specific event.
The event name (or activation name) is specific to each activation event and is unique. The goal of these changes is to allow for easier filtering when a specific component is clicked (TBD where this component will be).
Changes
For event name only
_parse_filenamewhich extracts disasters specific activation name and info from filename.discovery_itemsto allow for backwards compatibility"disasters:extract_event_name": true,. If the key/value pair is absent, airflow will run as normal. If the key/value pair is present and istrue, then the filename will be extractedevent:nameto the properties section for each item.For monty stac extension
discovery_itemsto allow for backwards compatibility"disasters:monty": true,. If the key/value pair is absent, airflow will run as normal. If the key/value pair is present and istrue, then the monty stac spec info will be addedCan test locally with the following config
PR Checklist
terraform validateandterraform plan